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0.  INTRODUCTION 

This  paper  attempts  a systematic  study  of  spherically  invariant  jiro- 
cesses  (SIP's),  i.e.,  processes  whose  finite  dimensional  distributions  are 
mixtures  of  Gaussian  distributions.  Section  1 contains  the  basic  properties 
of  SIP's,  including  their  representation  in  terms  of  Gaussian  processes, 
which  are  used  throughout.  The  structure  of  the  nonlinear  space  of  a second 
order  SIP  is  considered  in  Section  2.  Section  3 solves  the  problem  of 
discriminating  between  two  second  order  SIP's,  and  Section  4 the  nonlinear 
estimation  problem  for  second  order  SIP's  and  in  particular  for  Gaussian 
processes. 

Our  basic  notation  and  terminology  is  as  follows.  X = (X^,  teT)  is 

a stochastic  process  defined  on  a probability  space  (II, 8, P).  T is  an  arbitrary 

index  set;  sometimes  it  is  taken  to  be  a real  line  interval,  but  since  this  is 

clear  from  the  context  it  is  not  emphasized.  8 is  usually  taken  to  be  8fX) , 

the  a-field  generated  by  the  random  variables  of  the  process  X,  or  H(X) , the 

completion  of  8(X)  with  respect  to  P.  For  each  well,  X(w)  is  the  corresponding 

T 

sample  path  of  the  process  X,  which  is  an  element  of  IR  , the  space  of  all 

functions  defined  on  T.  X induces  a probability  measure  y = P'X  ^ on 

j.  T T T 

(K  ,8(IR  )),  where  8(IR  ) is  the  o-field  generated  by  the  cylinder  sets  of  IR  . 

X is  a coordinate  process  if  (II, 8, P)  = (IR^,8(1R ,y)  and  X^(w)  = w(t) . The 

. I nonlinear  space  of  X,  L-(X)  = L2(H,8(X) ,P) , is  the  set  of  all  8(X) -measurable 

- 1 

■ 3 random  variables  with  finite  second  moment  which  are  called  (nonlinear)  L^- 


functionals  of  X.  X is  a second  order  process  if  EX^  < for  all  teT.  The 
linear  space  of  a second  order  process  X,  H(X),  is  the  closed  subspace  of 


F" 


1.  SPHKRICALLY  INVARIANT  PROCTSSITS  (SIP's] 


It  is  well  known  that  all  mean  square  estimation  problems  on  Gaussian 
processes  have  linear  solutions  and  that  Gaussian  processes  are  closed  under  | 

linear  operations.  Vershik  (1964)  showed  that  these  two  properties  do  not 
uniquely  characterize  the  Gaussian  processes.  They  do,  however,  characterize  i 

the  class  of  SIP's. 


Let  X = (X^,  teT)  be  a second  order  process  with  mean  m(t)  and  covariance 

function  r(t,s).  Then  X is  said  to  be  an  SIP  if  all  r.v.'s  in  H(X-m)  having 

the  same  variance  have  the  same  distribution.  Also,  X is  called  degenerate 

if  H(X-m)  is  finite  dimensional.  Since  the  nonlinear  structure  of  a non- 

degenerate  and  a degenerate  SIP  arc  considerably  different,  we  will  restrict 

our  present  investigation  to  the  nondegenerate  case;  the  finite  dimensional 

case  will  be  treated  elsewhere.  A SIP  X is  a mixture  of  Gaussian  processes. 

It  can  be  determined  by  its  mean  m(t) , a covariance  function  R(t,s),  and 

a probability  distribution  F(a)  on  IR^;  the  characteristic  function  of 

,...,X  (t, , . . . ,t, eT)  is  given  by 

- j IR(t.,t  )(u.-m(t.))(u.-m(t.)) 

(1.1)  /e  1 J 1 1 J J 

(Vershik  (1964)  and  Nagornyi  (1970)).  In  order  to  avoid  the  trivial  case 

where  X is  a constant  process,  we  will  assume  throughout  that  F(O^)  < 1. 

Such  an  SIP  determined  by  m,  R and  F will  be  denoted,  in  short,  by  SlP(m,R;F). 

T T 

A probability  measure  p on  the  sample  space  (IR  ,B(IR  ))  is  said  to  be  a 

spherically  invariant  measure  (SIM)  if  it  is  induced  by  a SIP,  or,  equiv- 

t T 

alently,  if  the  coordinate  process  on  ( IR  ,6(IR  ) ,u)  is  a SIP.  A SIM 
induced  by  a SIP  (m.  R ,F)  will  be  denoted  by  SIM(m,R;F).  When  F puts  all 
its  mass  at  tlic  point  1 the  SIP  (m,R;I)  and  the  SIM  (m,R;F)  become 
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(laussian  with  mean  m and  covariance  U;  we  will  use  the  notation  CiI’(ni,IO  and 
GM(m,R)  respectively. 


LEMMA  1.1  Let  p be  an  SIM  (m,R:FJ  and  for  each  a > 0 let  be  a 


GM(m,aR) . Then 


(1.2) 


U(rO  = /p^(F.)dF(a)  for  all  EcB(lR  ). 


T T 

Furthermore,  for  any  measurable  function  0 on  (IR  ,B(1R  ) ,p)  which  is  non- 
negative or  integrable  we  have 


(1.3) 


B0  = /E^0dF(a) 


where  E0  = /0dp  and  E^0  = JOdp^. 


Proof:  For  each  fixed  cylinder  set  E,  d^(E)  is  a measurable  function  of  a: 

and  the  family  of  sets  E such  that  ^^^(E)  is  ex-measurable  is  a 0-field.  There- 

fore,  b(j(E)  is  a-measurable  for  every  EeB(lR  ).  Thus  /p^(E)dF(a)  is  well- 

defined  and  is  easily  checked  to  be  a probability  measure.  The  characteristic 

functions  of  its  finite  dimensional  distributions  are  given  by  (1.1),  and  since 

T T 

they  uniquely  determine  a probability  measure  on  (IR  ,B(1R  )),  we  have  (1.2). 

(1.3)  holds  for  0 = Ip^  (EeB(lR^))  by  (1.2).  Hence  it  holds  for  simple 
functions  0,  and  thus  for  0 nonnegative  by  considering  a sequence  of  simple 
functions  increasing  to  0.  For  0 integrable,  consider  the  positive  and  nega- 
tive parts  separately.  [] 

It  follows  from  Lemma  1.1  that  if  X is  a coordinate  SIP  (m,R;F)  then, 
under  each  p^,  X is  a GP(m,aR).  The  following  theorem  generalizes  this  fact. 

THEOREM  1.2.  Let  \ be  a SIP  (m,R;F)  on  (fi,B(X),P).  Then  for  each 


n > 0 there  exists  a probability  measure  P^  on  (f^,B(X))  such  that  p^  = P^^'X 
is  a GM(m,aR).  Hence  under  each  P^,  X is  a GP(m,aR)  and  the  corresponding 
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formulae  (1.2)  and  (1.3)  hold. 


Proof:  Let  u be  the  STM  induced  on  ( lU  ,B(IU  ) ) by  X and  let  a - 0, 

be  nM(m,  R)  as  in  Lemma  1.1.  Let  CCD^S  be  the  field  of  all  cylinder  sets 
in  B(R^)  and  C(X)  = X ^•C(IR^)  c 8{X).  For  each  DeC(X),  define  = 

p (E)  if  D = X ^(E).  We  claim  that  P is  well  defined.  It  is  not  hard  to 

a CL 

see  (by  looking  at  their  characteristic  functions)  that  when  restricted  to  a 
finite  dimensional  space  p^  is  absolutely  continuous  with  re.spect  to  p.  Thus 
if  D = X"^(E)  = X'^(E'),  E.E'eC(IR^),  then  p(E)  = P(D)  = p{Ej)  and  hence 
p^(E)  = p^(E').  Consequently,  is  a probability  measure  on  the  field  C(X) 

and  hence  it  has  a unique  extension  to  a probability  measure  on  8(X) . Now 

-1  T 

it  follows  that  p^  = P^^'X  since  the  two  measures  coincide  on  C(1R  ).  0 

(1.2)  in  Theorem  1.2  was  proved  by  Gualtierotti  (1974)  for  P an  SIM  on 
a separable  hilbert  space.  Note  that  since  X is  of  second  order,  it  follows 


” > fX“  = /E^X^dF(a)  = /[m“(t)  + aR(t ,t) ]dF(a) 
Oj  = JadF(a)  < . 


THEOREM  1.3.  Lot  X be  a SIP  (0,R;F)  and  <3  sequence  in  H(X).  Then 

there  exist  versions  of  the  such  that  under  each  P^  the  r.v.'s 

jointly  Caussian  with  zero  mean  and  covariance 


(1.4) 


E f.  E = EE  ^ = aE,E  E 

a n m n m 1 n m 


and,  moreover,  for  any  measurable  function  g on  ( IR  ,B(1R))  we  have 


(1.5) 


.^2’  • • - ^ = Eig(*''u  Ej , Et.  • • •) 


whenever  one  of  these  expectations  exists. 


Proof:  Tn  view  of  Theorem  1.2  arul  the  fact  that  every  measurable  function 


OCoj)  on  is  of  the  form  f(X(u)))  for  some  measurable  function  f on 

T T 

(IR  ,B(1R  ),  it  suffices  to  prove  the  assertion  for  X a coordinate  process. 

For  each  r.v.  xeIR  there  is  a sequence  (x)  of  finite  linear 

combinations  of  {f^(x)=x(t)  ,teT}  such  that  = lim^  (x)  a.s.  [p). 

Let  C = {xeR^:  £.^^Vx)  • as  i and  let  C =nC  . C is  clearly  a 

n 1 n 

measurable  linear  space  having  y probability  1.  Now  take  each  to  be 
lim.  £f'^|[xl  for  xeC  and  0 for  x^C.  Since  1 = y(C)  = /y^(C) dF (ex')  by  Lemma 
1.1,  and  F(O^)  < 1,  there  is  > 0 such  that  y (C)  = 1.  Let  Y be  a G(0,R) 
on  some  probability  space  (fl^.B^.Q).  Then  since  C is  a linear  space  we  have 
for  all  a > 0 


y^(C)  = Q(v^  YeC)  = YeC)  = y^  (C)  = 1 . 


Thus  = lim.  £^^^  a.e.  fy  1,  for  all  a > 0.  Since  under  each  y , 
n 11  ' a 

{jJ^^n^l , i>l}  is  a Gaussian  family,  it  follows  that  are  jointly 

Gaussian  under  each  y . 

a 

To  show  (1.4)  we  recall  that  for  a Gaussian  family  a.s.  convergence  is 
equivalent  to  mean  square  convergence.  Thus 


E f,  ^ = lim  E 

a n m . . a i j 

1 . J 

= lim  aE  £f"^£f"’^ 

1 1 j 

• ..1 


aEi^i^i 


a, 

and  hence  E^-E-  = /E  E.E-dF(cx)  = a.E  E,.^.  = — E E-E-- 
1 j •'  a 1 l^i^j  a a 1 j 

It  is  easy  to  see  that  under  y^  has  the  same  probability  law  as 

{/a  E } under  y,  . (1.5)  is  now  evident.  fl 

n 1 

tVe  now  introduce  a r.v.  A which  will  play  a central  role  in  the  repre- 
sentation of  a SIP  as  well  as  in  the  study  of  its  nonlinear  space. 
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Pick  an  orthogonal  sequence  from  H(X)  with  • Then 

by  Theorem  1.3  we  may  assume  that  under  each  1’^,  {Ej^}  is  a sequence  of 

2 

independent  zero  mean  Gaussian  r.v.'s  with  = a.  Define 

A = - I . 

n n j 1 

By  the  law  of  large  numbers,  we  have  lim  = a a.s.  [P^] • Pet  C*  = 

{coefl:  A (oj)  ^ •}.  Then 
n 


P(C*)  = /P^(C*)dF(a)  = JdF(a)  = 1 


Thus  A^  converges  a.s.  [P] . Let 


(1.6) 


A(w)  = 


lim  A (oj)  if  coeC* 
n 

0 if  ojEC* 


(1.7) 


A = a a.s.  fP  1 , 
' a' 


and  the  distribution  function  of  A is  F since 


(1.8) 


P(A<a)  = /P^(A<a)dF(a)  = /l (a)dF(a)  = F(a) 


If  we  put  9,^  = {toeSl;  A(u))  = a}  then  Pj^(l^^)  = 1>  and  thus  the  probability 


measures  P^,  a > 0,  are  mutually  singular  (which  is  of  course  well  known) 
We  now  arrive  at  the  main  theorem  of  this  section. 


THEOREM  1,4.  A nondegenerate  second  order  process  X on  (fl,B(X),P)  is  a 
SIP  (0;R;F)  if  and  only  if  it  has  the  representation 


(1.9) 


X^  = A ' a.s.  for  all  teT 


where  ^ is  a nonnegative  r.v.  with  distribution  function  F and,  conditioned 


on  A > 0,  V is  a nondegenerate  GP(0,R)  independent  of  A, 
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Proof;  The  "if"  part  is  clear.  To  show  the  "only  if"  part,  let  A be  the 
r.v.  defined  by  (1.6)  and  let 

/ 

X 


(1.10) 


A 


172 


0 


if  A > 0 
if  A = 0 . 


1/2. 


a > 0 


Then  it  follows  from  Theorem  1.2  and  (1.7)  that  P(X^  = A Y^)  = 
|p^(X^=a^^^Y^)dF(a)  = 1;  and  that  Y is  a GP(0,R)  under  each  P^  for 
and  thus  also  under  P(*)  = (1-F(0'^))  P (*)dF(a),  which  is  just  the 

(0,-)  “ .j. 

conditional  probability  of  P given  A > 0.  Now  for  DeB(IR)  and  F.e8(K  ) 
we  have 

P(AeD,  YeE)  = / l„(a)P  (YeE)dF(a) 

1-F(0  ) (0.c») 


1 


1-F(0  ) 


P(AeD)  • P(YeE) 


= P(AeD)  P(YeE) 

since  P^(YeE)  = P(YeE)  for  a > 0.  Hence  A and  Y are  conditionally  independent 


given  A > 0. 


□ 


, I 


■ 1 


The  representation  (1.9)  of  a SIP  was  first  noted  by  Besson  (1974)  who, 
without  constructing  the  r.v.  A,  showed  its  existence  by  employing  a result 
in  Bretagnollc  et . al  (1966)  concerning  s>mimetrical ly  dependent  (exchangeable) 
r.v.'s.  Our  approach  seems  more  elementary  and  direct,  and  also  yields 
further  results. 

(1.9)  reveals  that  a SIP  is  merely  a conditional  Gaussian  process. 

(More  specifically,  given  A = a,  the  SIP  (0,R;F)  is  a GP  (0,aR).)  By 
requiring  F(O^)  = 0,  Theorem  1.4  may  be  stated  in  a more  appealing  way. 
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. f 


I 


THEOREM  1.5.  A nondoqenora  te  second  order  process  X on  (f2,8(X),l’) 

+ , 


is  a SIP  (0,R;F)  satisfying  F(()  ) = 0 if  and  only  if  it  has  the  repre- 
sen ta t i on 


1/2 

X^  = A ' a.s. , teT 


where  f is  a nondegenerate  OP  (0,R)  and  A is  a positive  r.v.  independent 
of  Y having  distribution  function  F. 


Theorem  1.4  enables  one  to  read  off  many  properties  of  SIP's  immediately. 
For  instance  a SIP  (0,R;F)  X is  of  order  p > 0,  i.e.,  E|X^P  < «>,  if  and 
only  if  EA^^^  = /a^'^^dF(ot)  < “■.  If  X is  of  order  2,  then  continuity  in 
probability  of  X is  equivalent  to  mean  square  continuity  [Besson  (1974)]. 

All  the  usual  (local  and  global)  analytic  properties  of  the  sample  functions 
of  X depend  only  on  R and  not  on  F,  while  properties  of  maxima  and  crossings 
depend  on  both  R and  F.  Kallianpur's  zero-one  law  and  Slepian's  lemma 
take  the  following  form  for  SIP's.  (The  proofs  are  straightforward  and 
are  thus  omitted.) 


COROLLARY  1.6.  If  p is  an  SIM  (0,R;F)  and  L a B(IR  ) -measurable  linear 


subspace  of  then  y(L)  = F(O^)  or  1.  Furthermore,  p(L)  = 1 if  and  only 


if  for  some  a > 0,  L is  8 (IR  )-measurable  and  u (L)  = 1. 

a a 


— T — T 

In  Corollary  1.6  are  as  in  Lemma  1.1  and  8(11  ),  resp.  8^(1R  ). 


denotes  the  completion  of  8(1R  ) with  respect  to  p,  resp.  p^. 


COROLLARY  1.7.  If  X.  is  a separable  SIP  (0,R.;F.),  i = 1,2,  and  if 


Rj(t,t)  = R2(t,t) 


F^(0^  = F2(0*) 


Rj(t,s)  < R,(t,s)  for  all  t,scT  , 

Fj(a)  < p2(a)  for  all  a > 0 , 


then  for  all  u. 


-.*v. 


J 
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P{sup  X.  (t)  < u}  ^ P{sup  X^(t)  < u) . 
teT  tcT  " 

Wc  now  give  an  interesting  example  of  a sample  continuous  martingale 

whose  family  of  a-fields  is  not  continuous.  Let  W = 0<t<“)  be  a 

1/2 

Wiener  process,  and  let  X = A W where  A is  a nonnegative  r.v.  independent 
of  W and  whose  o-field  is  nontrivial.  It  is  easily  checked  that  X is  a 
sample  continuous  martingale.  Let  8^  = ^(X^,,  0<s<t)  , 0<t<«>.  We  will  show 
that  8^  is  not  continuous  att=0.  Fixt>0  and  consider  the  quadratic 
variation  M(t)  of  X over  the  interval  [0,t].  We  have 

M(t)  = lim  I fX(i^  t)  - X(-^  t)]^ 
n j=l  2 2 

2n-l 

= A lim  I [W(i^  t)  - W(-4  t)l^ 
n j=l  2 2 

= At  a . s . 

where  the  last  equality  is  a theorem  of  Levy  [Doob  (1953)].  This  implies 
that  A is  8^-measurable  for  all  t > 0. 


^ . 


2.  THI-:  NONLINEAR  Sl’ACH  OF  A SIP. 


1 J 


In  this  section  we  study  the  structure  of  the  nonlinear  space  L.^(X) 
of  a SIP(0,R;F)  X,  using  the  canonical  representation  of  X in  Theorem  1.4 
and  the  well  known  properties  of  the  nonlinear  space  of  a GP.  When  X is 
second  order,  the  relation  between  the  linear  space  H(X)  and  the  nonlinear 
space  L^(X)  is  shown  in  Theorem  2.4  and  complete  orthonormal  sets  (CONS' s) 
in  given  in  Theorems  2.2  and  2.3.  When  F has  a moment  generating 

function.  Theorem  2.5  shows  that  L^CX)  has  the  orthogonal  decomposition 

where  Hp(X)  is  the  p-th  homogeneous  chaos  of  X;  and  Theorem  2.6 
shows  the  relation  between  H^CX)  and  H(X)  and  gives  CONS's  in  each  H^CX). 

Theorems  2.2  to  2.4  are  based  on  the  following  property. 

LEMMA  '' . 1 Let  and  be  two  independent  0~ fields  on  a probability 
space  (il,  B,  P)  such  that  B is  generated  by  B^  and  B2 . Then 
L^{P..B,P)  = L^fJl.B^.P)  B L2(J2,B2,P) 
under  the  correspondence  fg  f « g 


Proof:  We  will  write  l..,(8.)  for  L.(C;,B.,P).  Consider  the  mapping  taking 

f s g to  fg  for  all  fel^CBj),  geI..,(B^).  From  the  independence  of  8j  and  B., 
it  follows  that  fg  e L^CB)  and 


< fj8).  8282  * LjtB) 


Thus  the  mapping  preserves  inner  products  and  it  has  a unique  extension  to 
.'in  isomorphism  between  the  closed  subspace  spanned  by  {f®g},  which  is  in  fact 
« I.-,(8t),  and  the  closed  suhspace  of  L2(B)  spanned  by  {fg}.  The 
assertion  is  proved  if  we  show  that  M = f fg  :f  eL,  ) , geL.,(82)}  is  a complete 
set  in  1,,(B). 
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Consider  the  family  6*  of  sets  of  the  form  u?(F;.nF,).  b.eB,  and 

Since  and  are  fields  and  they  generate  8,  it  is  readily  seen 

that  B*  is  a field  generating  B.  We  may  even  assume,  after  a moment's 

reflection,  that  the  sets  E.nF.,  1 < i < n,  are  disjoint.  Now  a standard 

11 

monotone  class  argument  shows  that  the  set  {l  n,„  x Ir-  } 

utE.nl.)  ‘'lE.F. 

T '■  T -I  •'  1 I 


r 1 1^ 


1 1 


is  complete  in  L^CB).  Thus  M is  complete. 


□ 


Let  X = {X^,teT}  be  a second  order  nondegenerate  SIP(0,R;F)  and  X = A*Y 
its  canonical  representation  of  Theorem  1.4.  Let  linearly  ordered) 

and  (e^,  l^n<N}  (N  may  be  infinite)  be  CONS' s in  H(X)  and  L^tdF)  respectively. 


THEOREM  2.2  I f F (0  ) = 0 then  the  family 

n ^^!...pj^!  p^  A Yj  Pk  ^ 


1 J- 


= H A )...H  A (C  ) 

n Pi-'-.-Pj,!  A pj,—  Yj  Y,^ 

where  1 <nSN,  k>l,  ?!■*■•••  ~ P - Yj,  <...<  Yj^  is  a CONS  in 

L2(X) 

Proof : Assume  F(O^)  = 0.  Then  8(A)  and  8(Y)  are  independent  a-fields  generating 

B(X).  By  Lemma  2.1  we  have 

(2.2)  L^tX)  = L^tA)  B L2(Y)  . 


I 01, 

It  is  easily  verified  that  A^l(Y)  = H(X)  and  {p  = (t— ) ^ C ,YeF}  is  complete 

Y A Y 

in  H(Y).  (Recall  that  Y is  a GP.)  Indeed  {p.^,YeP}  is  a CONS  in  H(Y)  since 
a a. 

E(^  ^ Cn)  = i — E ^ C„dF(a)  = E C C- 
A Y B ■'  01  a Y B Y B 

(by  Theorem  1,5).  Now  (2.2)  and  the  celebrated  theorem  of  Cameron  and  Martin 
yield  the  result.  0 


^ TT 


We  remark  that  in  dealing  with  a SIF,  the  case  F(0+)  = 0 
(i.c.  X cannot  be  zero  with  positive  probability!  is  of  main  interest. 

We  now  show  how  to  obtain  the  counterpart  of  Theorem  2.2  when 
0 < F[0+}  < 1.  Let  = {u)e52  : A(o))  = 0),  = {o)cf2:  A(to)>0},  and 

consider  the  restrictions  of  (ii,8(X),P)  to  and  respectively, 

’I’"" 
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It  is  easily  seen  that 


where  IR  is  equiped  with  the  usual  inner  product  (<x,y>^=  xy) . Also 

IK  ^2 

where  Q is  defined  on  BCX)  by  Q(o)  = j ^^P^(°)dFCa) , and  thus 


. I 


under  the  correspondence  (1-F(0+))"^  1^  f +-+  f.  But  under  P , X is  a 
SIP(0,R;G)  where  G(a)  = (1-F (0+) ) "'^(F (a)-F C0+) ) , and  Theorem  2.2  yields 
the  following  CONS. 


TIEOREM  2,3  If  0 < F (0+)  < 1 and  if  {e^,  l<n<N}  is  a CONS  in  L.,  (d{F-F(0+)  ]) 
then  the  family 


P i 


*-F(0+)^  ^ ^{0}*^'''^^' 


(2.3) 


( 


1-F(0+)  \o,oo) 


(A)  eJA)  (— 


I-  0 

2 r ^ 


‘l 


where  1 < n < N,  k > 1,  p^  + .-.+p^  = P > n,  • < Yj^  F'  is  a CON  S in 


4(x). 


•*.*•»*•  **»  ^ 

^ 
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Consider  the  correspondence 


a. 


e (A)  (—7-^ r)^(7^)^  '<  A (f.  ) 

n p,!...p '■A  ' p 


(2.4) 


'p,  ! . . .p,  ! ' "A  " "p 

1 * k * I »a  '1 

1 


"p  ^ ^ 

Pk'u^  'k 


e^(A) 


C_L’J 


s 


Pk 


when  F(0+)  = 0,  and  when  0 < F(0+)  < 1 the  correspondence 


^ ’■F(0+)^  ^{0}*-'^^’ 


(2.5) 


1.2. 


p.  % ®P1  ^ ~ ®pk 

Pi Pk'  '1  \ 


pi  ^ ®Pl  ~ ~ ®Pk 

Since  (-^  ;_.p"T)  where  k > 1,  p^  +...+  pj^  = p > 0, 

1 k 1 k 

s^p 

Yj^  <.  ..<  Yj^>  form  a CONS  in  H (X),  we  have 


THEOREM  2.4 

~ ®P 

L (X)  = L CdF)  ® C ® H (X)) 

P ^0  ~ 

®p 

= L (A)  ® ( ® H ' (X)),  if  F(0f)  = 0 ; 

p>0 

~ ®P 

L (X)  = ]R®{L  (d(F-F(C+))  ® ( ® H (X))},if  0 < F(O^)  < 1. 

p>0 


Suppose  that  every  ^€H(X)  has  all  moments  finite.  Let  P(X)  be  the 
linear  space  of  all  polynomials  in  elements  of  H(X)  and  let  Pp(X)  (p>0) 
be  the  linear  space  of  all  polynomials  in  P(X)  of  degree  at  most  p;  hence 
Pq(X)  is  the  set  of  all  constants.  Let  Qp(X)  = P“^ 

Qp(X)  be  the  set  of  all  polynomials  in  P^(X)  orthogonal  to  j(X).  Denote 


r 
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by  Hp(X)  the  closure  of  Q^(X)  in  L^CX).  Q^(X)  is  called  the  p-th 
polynomial  chaos  and  ll^(X)  is  called  the  p-th  homogenous  chaos. 

When  X is  a S1P(0,R;F),  in  order  to  have  all  moments  of  CeH(X)  finite 
we  introduce  the  following  "moment"  condition: 

(M)  The  moment  generating  function  of  F exists,  i.e. 

[ Ott 

j e dF(a)  < «>  for  all  telR  . 

Under  the  condition  (M3  we  have  for  CeH(X)  (by  (1.5)3 , 

= j dF(ot3  = a^’^^FjClPdr(a3  = ^ ” 

f D 

where  = a^dF(a),  p>0. 


THEOREM  2.5  If  (M)  holds. 


L (X)  = © H (X). 

p>0  ^ 


Proof:  It  is  well-known  (see  Neveu  (1968)3  that  if  for  every  CeH(X) 

then  L (X)  = © H (X)  (for  arbitrary  process  X).  Thus  it  suffices  to  show  that 
p^O  P 

e^  is  integrable  if  (M)  holds.  Since  under  each  P , 5 is  a zero  mean  Gaussian 

Ot 

2 a ^ 

variable  with  variance  a - — 5c  , we  have 

* 

r - 2 f ^ x-(x^/2o2) 


/Tn  n •'0 


/2tt  a •'  -o 


2 2 2 
e-^/2-  dx  s 2e"/' 
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Thus,  under  (M), 

E.l«l  . 


E 


dF(a) 


< 2 


2a  ^ 

e dF(a)  < “ . 


0 


We  now  establish  the  relationship  between  the  decomposition  of  Theorem  2.5 

and  the  representation  of  Theorem  2.4.  Assume  the  condition  (M) . Then 

L-CX)  = e ^r.(X),  and  {a",  0<n<“}  is  a complete  set  of  L-(A).  For  each 
p>0  P ^ 

Fixed  q > 0,  applying  the  Gram-Schmidt  onthonormalization  procedure,  to  {A*^} 

with  respect  to  the  inner  product  <a"’,  = E we  obtain  the  set 

(e^(A),  0<n<®}  which  is  complete  in  L2(A)  and  orthonormal  relative  to 

<*,*>^  . Note  that  e^(A)  is  a polynomial  in  A with  degree  n. 

Now  assume  F(0+)  = 0.  By  Theorem  2.4  we  have  L (A)  « ( ® H^^CXJ)  = L-(X). 

^ p>0  ^ 

Denote  this  isomorphism  by  4>.  Let 

(2.6)  = <P(eJ(A)(^)‘’/^  ® H^(X)  } • 

It  is  not  difficult  to  see  from  (2.4)  that 


(2.7) 


^ "’2n.qf^^  ^ "’2n.q-l 


(X) 


since  A e P^,  from  the  definition  of  A. 


THEOREM  2,6  Under  conditions  (M)  and  F(0+)  = 0, 


(2.8) 


«pf^^  % * »n,qf^) 

^ 2n+q=p 


and  a CONS  in  Hp(X)  is  given  by 


(2.9) 


^q  ! - - . n ! 


1 .q, 


„ ,)  e^(A)  H A_  (£  )...H  ) 

" ‘^I’aj  Yj  qR'a^  \ 


where  2n+q  = p,  k > 1,  q^  + ...+q|^  = q,  Yj<.-.<Y^. 
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Proof : First  we  show  that  ^(X)  i (X)  if  (n,q)  / (n',q').  It  is 

clear  from  (2.6)  that  ^(X)  i H^,  (X)  if  q q'.  Suppose  now  that 

q = q'  and  n ^ n'.  For  0,  9'  *:  H*^(X)  we  have 

E{  <I>(e^(A)(^)‘^/^  ® 9)  $(e^,(A)(^)'*/^  ® 0')  } 


= -Jf{e^(A)e^,(A)A‘l}  <0.9’>h^(X)  = ° 


.q  n'  n 


since  <e*^,e^,>  = 0 and  thus  H (X)  i H , (X) . 

n n'  q n,q  n'q' 


In  order  to  show  (2.8)  it  suffices  to  show  that  P (X)  = * H (X) 

p _ n,q 

^ 2n+q<p 

because  of  (2.7).  We  need  to  show  that  e ® H (X)  for  all 

2n+q<p 

e H(X)  and  r^+. . .+Tj^  < p.  For  ease  of  exposition  we  show  this  for 
only.  Write  5 = A^n,n  e H(Y).  Then 


_(n)  + Const.  H ,(n)  + Const.  H ,(n)  + ...} 

1 r.En  r-2,En'^  r-4,En'^ 

. I 

= H 2*^^^  Const.  A H (Q  + Const.  A“H  , (C)  + ... 

r,AEq  r-2,AEn"^  r-4,AEn^ 


III  T* 

Note  that  AH  -(D)  e ® H . (X)  c e H (X).  Thus  K € ® 

r-2m,AEn  n=0  2n+q<r  2n+q<p 

I p(X)  and  (2.8)  is  proved. 

(2.9)  follows  from  (2.4),  (2.6),  (2.8)  and  1 le^(A)  (— )^^^  1 1 ^ = a.^^.  0 

I n otj  1 

I Of  course  (X)  = H(X)  and,  for  instance  H2(X)  = ^(X)  © 2^^)  where 


H,  „(X)  = sp(A-a. } and 
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3.  liQIHVAU-NCi;  AM)  SINQILAR/TV  Ol-  .SIP's 

In  this  section  we  combine  the  representation  of  SIP's  (Theorem  1.4)  and 
the  dichotomy  of  GP's  to  the  problem  of  discriminating  between  two  SIP's. 

The  discrimination  problem  is  completely  solved  by  identifying  the  Lebesgue 
decomposition  of  the  distributions  of  the  two  processes  and  the  Radon -Nikodvm 
derivative  of  the  absolutely  continuous  part.  Two  processes  are  called  equiv- 
alent  (~) , resp.  singular  ( i) , if  their  induced  measures  on  B(IR  ) are  equiv- 
alent (~) , resp.  singular  ( i) . The  discrimination  problem  for  two  second  order 
SIP's:  SIP(0,m;F)  and  SIP(m,S;G)  is  fully  resolved  by  first  noting  that 
without  loss  of  generality  we  may  assume  that  either  GP(0,R)  ~ GP(m,S)  or  else 
GP(0,aR)  1 GP(m,BS)  for  all  a,B  > 0.  When  GP(0,R)  ~ GP(m,S) , the  Lebesgue 
decomposition  of  SIM(m,S;G)  with  respect  to  SIMfO,R;I')  is  given  in  Theorem 

3.1  and  the  Radon-Nikodym  derivative  of  the  absolutely  continuous  part  in 
Theorem  3.4.  In  particular,  if  GP(0,R)  ~ GP(m,S),  then  dF  ~ dG  implies 
SIP(0,R;F)  ~ SIP(m,S;G),  and  dF  i dG  implies  SIP(0,R;F)  i SIP(m,S;G).  Theorem 

3.2  shows  that  if  GP(0,aR)  i.  GP(m,6S)  for  all  a g > 0 then  SIP(0,R;F)  i SIP(m,S;G). 
Finally,  if  dF  ~ dG  then  SIP(0,R;F)  and  SIP(m,R;G)  are  either  equivalent  or 
singular,  and  necessary  and  sufficient  conditions  for  equivalence  along  with 

an  expression  for  the  Radon-Nikodym  derivative  are  given  in  Theorem  3.5. 

For  reasons  of  clarity  in  this  section  we  attach  the  underlying  probability 
measure  to  the  usual  notation  for  expectation,  variance,  linear  space,  etc. 

We  first  state  the  general  theorem  concerning  the  equivalence  and  the 
singularity  of  two  GP's  (see  for  instance  Pang  (1973)).  Fix  a > 0.  Let 
X = (x^,  tcT)  be  a GP(0,aR)  on  the  probability  space  (0,B(X),P  ).  Let  Q 
be  a second  jirobability  on  (il,B(X))  under  which  X is  a GP(m,aS).  Then  cither 
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P ~ 0 or  P i 0 . P ~ Q if  and  only  if  the  following  conditions  are 
a a a a a 

satisfied: 

(*)  There  exist  positive  constants  and  such  that 

Varp  C ■'  VarQ  C ■'  Kj  Var  p C 
a a 

■ • V 

for  every  CeL(X) , the  linear  space  of  all  finite  linear  combinations 

n 

t eT. 
n 

(**)  There  exists  Cell  (X)  such  that  m(t)  = Ep  (EX^),  tcT;  i.e.  meR(R), 

a 

the  reproducing  kernel  Hilbert  space  of  the  covariance  R. 

(***)  If  B : Hp  (X)  Hp  (X)  is  the  positive  self-adjoint  operator  defined 
a a 

by  Cov  (C.P)  = CoVp  (C,B  n)  for  all  E>  neL(X) , then  B^  - I is  Hilbert-Schmidt . 
^ a 

Moreover  if  conditions  (*) , (**)  and  (***)  hold  true  and  if  and 

denote  the  sets  of  eigenvalues  and  corresponding  normalized  eigenvectors  of 


B , then 
a’ 


(3.1) 


= exp{B;^/^C  - y Varp  -D  - logAj} 


From  now  on  X = (X^,  teT)  will  be  a second  order  nondegenerated  SIP 
(0,R;F)  on  (J1,B(X),P).  Recall  that  R is  a covariance  function  and  F is  a 
distribution  function  on  IR  with  finite  first  moment  a^.  P is  a mixture 
of  Gaussian  measures,  P(E)  = /p^fE)dF(a),  and  under  each  P^,  X is  a 
GP(0,ntR).  Let  A be  the  r.v.  associated  with  X and  P as  in  Section  1. 

Now  consider  a second  probability  measure  Q on  (fi,8(X))  under  which  X 
is  a second  order  nondegenerate  SIP  (m,S;G).  Then  Q(F)  = /Q^(F)dG(a)  and 
under  each  Q , X is  a GP(m,aS).  Denote  the  first  moment  of  G by  a| . We 
arc  interested  in  the  equivalence  and  mutual  singularity  of  the  measures  P 
and  Q.  Since  i if  m 0,  we  shall  assume  throughout  this  section  that 


V 
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F(0+)  = 0.  Also  wc  may  assume  without  loss  of  generality  that  either  I’j  ~ Qj 
or  0.  Qg  for  all  a, 3 > 0,  since  clearly 

P.2)  a SIP  (m,R;F(a)),  is  also  a SIP  (m,cR;F(ca)l  for  every  c > 0 . 

THEOREM  3.1  Let  Pj  ~ Q|  = dG ' + dG"  be  the  Lebesgue  decom- 

position of  dG  with  respect  to  dF  with  dG'  dF  and  dG"  i dF.  Then 

Q(fO  = /Q^(K)dG'(a)  + /Q^(F.)dG"(a) 

is  the  Lebesgue  decomposition  of  Q with  respect  to  P.  Hence  if  P^  ~ Qj  and 
dF  ~ dG  then  P~Q;  ifPj~Qj  and  dF  1 dG  then  P X Q. 

Proof:  Note  that  P.  ~ Q,  implies  P ~ Q for  all  a > 0.  Let  P(E)  = 0.  Then 

/p^(E)dF(a)  = 0 and  P^(E)  = 0 [dF] . Since  P^  ~ and  dG'  <<  dF,  we  have 

Q (E)  = 0 a.e.  [dG'j,  and  thus  /Q  CE)dG'(a)  = 0.  This  implies  /(T  (*)dF(a)<<P(*) 
Since  dG"  i dF,  there  exists  Ee8(IR)  such  that  Jj;dG"(a3  = /^,dF(a)  = 0. 

Note  that  A = a a.e.  [Q  ] because  A = a a.e.  [P  ] and  P„  ~ C)  • Thus  we  have 

P(AeE')  = /P^(AcE')dF(a)  = /j,,dF(a)  = 0 , 


. I 


"I 

i 

, 1 


/Q^(A£E)dG"(a)  = JpdG"(a)  = 0 , 


which  imply  /Q^(*)dG"(a)  x P(*) 


The  second  assertion  of  Theorem  3.1  was  first  stated  in  Gualtierotti 
(1974i  for  P and  Q SIM's  on  a separable  Hilbert  space. 


n 


THEOREM  3.2  If  P X Q for  all  a,6  > 0 then  P X Q. 
a g 

Proof:  The  proof  is  adapted  from  Pang  (1973). 

The  following  remarks  will  be  used  without  further  comment. 
?;,ncL(X),  we  have 


For 
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(3.31 


Ep  ^ = ^P^  = ^ = ~Covp(C,n)  , 

a a 1 


If  and  Qj  .'satisfy  {*)  then  for  every  sequence  in  Up  (X)  there  exist 

versions  of  ^ ’s  such  that  {f]  } is  also  a sequence  in  llp(X),  H (X),  Hq(X)  , 

Hq  (XI  for  all  a > 0,  and  satisfies  (3.3).  (The  proof  of  this  is  similar  to 
that  of  Theorem  1.3). 

Since  Pj  i , one  of  the  conditions  (*) , (**)  and  (***)  must  be  violated. 

First  suppose  that  (*)  is  not  satisfied;  for  instance,  suppose  that  there 

exists  no  constant  such  that  Varg  C < Var^  C-  Then  there  exists  a 

sequence  (C  } in  L(X)  such  that  Varr,  C = 1 and  Var^  C > n . Then,  as  n 
n ^1  ^1 

we  have 


P(  I I ^ fn)  ^ 0 . 


Qd^nl  > v^)  = /Qc,(Unl  ^)dG(a) 


J ~ f .I/O 


e dx)dG(a) 


-(an) 


which  imply  P i Q. 

Next  suppose  (*)  holds  but  (**)  does  not  hold.  Then,  for  each  n,  there 

exists  ^ cL(X)  such  that 
n 

fq/-n  " <"  2 (K^n 

(see  Pang  (1973)).  Consequently,  as  n -*■  “ 

’’'«n  ’ ^ ’’•f-n  * ^ - • 


’ K,'n>  = «l«n-EQe„l  < 


> 1 L ->  1 

n 
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•.  I 


1 


and  thus  P i Q. 

finally,  suppose  that  (*)  and  (**)  are  satisfied,  but  (***)  is  not.  We 
may  assume  in  this  case  that  m ? 0.  We  claim  that 

there  exists  a sequence  in  11^,  (X)  such  that  CoVp  (C- ,C.) 

(3.4)  = «..  and  Cov^^rq.tp  = UjS.j  wLe  I(J-U„)"  - - 'and 

^ > 0 for  all  n or  < 0 for  all  n. 

n 

Given  this,  consider  the  events 


<(jr  - ■)  4 i - d„ 

^ ^ In 


1 ^ 1 
A 1 


)}  . 


We  shall  deal  with  the  case  — 

% 

For  a > 1 we  have 


- M 


> 0 only.  Note  that  -<  by  (*] 


^^'m^  ■ J - U )} 

1 n 1 

m 

= r.(II  (V'H^-i)l  ->5^I(i-u)h 


1 2 

a 


3 i;»-d„) 


and  for  B s 1 


" <JifI  <(5^  -1)  .<  il  (ji-  d„n 

1 n 1 ^n 

m 

■‘'('li  'VV(r-”i  '4-i 

1 '"n  "11  " 


8k: 


->  1 . 


(h-  if  . u«„„,  E,.  then  P^u:)  • 0 for  d , , „„d  g^,f,  . , for  6 t 1.  This 
implies  hy  that  for  all  a.b  > 0 there  exist  F.^|,ee(\)  such  that 
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P (E  . ) = 0 for  a S a and  Q„(E  . ) = 1 for  3 s b.  Hence,  as  a -*■  0 and 
ab-^  ^3  ab 

b -*■  we  have 

P(Eab^  = / P^(E  )dF(a)  < F(a)  ^ F(0+)  = 0, 

[O.a] 

[0,b]  ^ 

and  consequently  P i Q. 

To  complete  the  proof,  we  now  verify  (3.4).  Two  cases  are  to  be 

considered.  First,  suppose  that  - I is  compact  but  not  Hilbert-Schmidt . 

Let  {^j^}  and  be  the  ergenvalues  and  the  corresponding  normalized 

r 2 

ergenvectors  of  B,  - I.  We  have  A ^ 0,  )A  = «>  and  B,  n = (1+A  )n  . 

^ 1 n '-n  In^n-^n 

By  choosing  a suitable  subsequence  of  we  obtain  the  desired  sequence 

in  (3.4).  Second,  suppose  that  B^  - I is  not  compact.  Being  invertible 

(by  (*)),  B^  is  not  compact  and  thus  its  essential  spectrum  is  not  10}. 

Furthermore,  since  B^  - I is  not  compact,  there  is  at  least  one  point 

p 1 in  the  essential  spectrum  of  B^ , and  thus  also  in  the  essential 

numerical  range  of  B.  (For  a nicv  discussion  of  essential  spectrum  and 

essential  numerical  range  see  Fillmore  et.  al.  (1972)).  Now  by  a known 

result  in  operator  theory  (Lemma  2 in  Anderson  and  Stampfli  (1971)  stated 

for  a separable  Hilbert  space  but  true  for  a nonseparable  Hilbert  space 

as  well),  there  exists  an  orthonormal  sequence  {p^}  in  Hp  (X)  such  that 

Cov(n.,  Bn.)  = U.6..  and  u u . Again  {r  } in  (3.4)  is  obtained  bv  choosing 
^ 1 j 1 ij  n n 

a suitable  subsequence  of  D 

Theorem  3.2  is  not  true  in  general,  without  the  Gaussian  assumption  on 
P^  and  Qg,  since  there  are  uncountably  many  measures  involved.  For  example, 
if  P is  the  uniform  measure  on  [0,1]  for  each  a,  if  Q„  is  the  one  point 

01  p 

mass  at  3 for  each  3,  and  if  dF  ~ dG  ~ P, , then  P i Qo  for  all  a and  3, 

1 OL  p 


but  P ~ Q. 


24 


Now  suppose  that  Pj  ~ and  let  Q'(*)  = /Q^ ( • )dG' (a1  be  the  absolutely 

continuous  part  of  Q given  in  Theorem  3.1.  We  will  calculate  the  Radon- 
dO' 

Nikodym  derivative  . Tl\eorems  3.1  and  3.2,  together  with  the  expression 
dO' 

of  , provide  a complete  solution  to  the  problem  of  discriminating  between 
two  SIP's.  To  this  end.  we  prepare  the  following 

dQ 

cx 

Lemma  3 . 3 Let  a.s.  [P^]  • Tf  ^ measurable  function, 

then 

d^  (w)  = dF" 

Proof : For  every  E e B(X),  we  have 

/ Pa  W-  Pa  ^ 

E E 

= /Q^fE)dG-(a)  = Q'(E) 

as  required.  0 

Note  that  the  measurability  of  P..  . fo))  is  not  automatic  since  each  0 

A (05 ) a 

can  be  arbitrarily  changed  on  a set  of  P -measure  zero. 

a 


Theorem  3.4  Suppose  ~ ' 


Then 


1 


(i)  there  exists  ^cHpCX)  such  that  m(t)  = ^Ep(CX^),  teT; 

(ii)  there  exists  a self-adjoint  positive  operator  B on  H CX) 

P 


defined  by 


^ Cov^C^.n)  = — CovpCC.Bn) 

for  all  neLCX),  and  such  that  B - I is  Hilbert- Schmidt. 

Moreover  if  and  is  the  set  of  eigenvalues  of  B and  their  corresponding 

1/2 

eigenvectors  with  norms  , then 


(3.5) 


d^  A ^-1/2, 

^ = exp  B £ 


Var 

2a^A  p" 


OO  V 

ini  (jr-n*iogxj}  • (A)  . 


dF 
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1/2 

Proof : Let  X = A Y be  the  canonical  representation  of  X,  where  Y is  a 

GP(0,R)  under  P and  under  each  P (rx>0) . It  is  clear  that  each  (X) 

a I 

1/2 

or  lip  (X)  has  a version  of  the  form  A n with  n«Hp(Y).  In  this  sense,  we 

l/T 

have  Hp(X)  = A '‘"H(Y)  = 11^  (X).  Therefore  every  operator  on  H^fX)  induces 

a 

an  operator  on  lip  (X)  in  an  obvious  way,  and  vice  versa, 
a 

Since  P,  ~ Q, , there  exists  a ieiip  (x)  such  that  m(t)  = £„  (CX  ),teT, 

and  a self-adjoint  positive  operator  B on  IL  (X)  defined  by  Cov~  (5,n)  = 

CoVp  (C, B n) ,5,neL(X) . A simple  computation  shows  that  ^ satisfies  (i) 

1 

and  that  B,  the  operator  on  Hp(X)  induced  by  B^,  satisfies  (ii). 

By  Lemma  3.3,  (3.5)  will  follow  if  we  show 


dQ  1 1 / o 

^■7  ct  ,1  „-l/2,_ 

(3.6)  ^ = exp  {-  B _ 

a 1 


1 -1/2 
Var^  (B 
2a,  a P ^ 


I h-s  (r-‘)*iosV>  >''"1 

1 n 


a 


Let  B be  the  operator  on  Hp  (X)  induced  by  B.  Again,  a simple  computation 
“ -1 

shows  that  a £ satisfies  condition  (**),  and  that  B satisfies  condition 

a 

(***)  and  has  eigenvalues  and  corresponding  eigenvectors 

{a  }.  Thus  we  have  by  (3.1) 

n 


dQ 

dP 


;1  n-1/2.  1 ,,  , 0-1/2  , 

^ aa  ^ -2  P ^ a E,’ 

2a  a 


-T  ([  -D-logXJ)  a.s.[P^] 


which  is  eipiivalent  to  (3.6)  since  Bj^^'^^£  = B ^^^^£  a.s.  [Pq]  • 
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r ~ — 

[ 

I 

. Theorem  3.5  suppose  R = S and  df’  ~ dG.  Then  either  P ~ Q or 

I P i.  Q,  and  P ~ Q if  and  only  if 

' (3.7)  m(t)  = Ep(?X^),teT, 

i 

for  some  ^eHp(X).  In  the  case  of  equivalence, 

cx 

I (3.8)  §=  exp  (c4  VarpC)  . § (A) 

j 

Proof : Clearly  if  (3.7)  holds  we  have  ~ ^1  ’ (3-7)  does  not  hold 

we  have  P x Q„  for  all  a,g.  The  first  assertion  follows  then  from 
a p 

Theorems  3.1  and  3.2.  (3.8)  follows  from  Theorem  3.4  by  comparing  (3.7) 

with  (3.6)  and  noting  that  R = S implies  B = I.  □ 

Sytaya  (1969)  derived  (3.8)  for  P and  Q SIM's  on  a separable  Hilbert 
space  and  F = G. 


. I 
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» 4.  NONLINl^AR  KSTIMATION  AND  PREDICTION 

i 

E Using  the  tensor  product  structure  of  the  nonlinear  space  we  solve  the 

general  nonlinear  estimation  problem  for  SIP's,  and  in  particular  for  GP's, 
in  the  sense  that  we  reduce  the  nonlinear  problem  to  a standard  linear  estima- 
tion problem,  the  theory  of  which  is  well  developed.  Also  we  derive  a lower 
bound  for  the  mean  square  error  of  the  nonlinear  prediction  for  a certain  class 
of  prediction  problems. 

4 . 1 Nonlinear  Estimation 

Let  X = (X^,  teT)  be  a second  order  process  with  zero  mean.  Consider  the 
following  estimation  problem:  we  observe  X^  for  teS,  a subset  of  T,  and  we  want 

» 

j to  estimate  an  L2~functional  9 of  X based  on  the  observations.  We  are  interested 

/N 

in  finding  the  best  estimate  6,  an  L2"functional  of  (X^,  teS)  which  minimizes 

^ 2 

the  mean  square  error  of  estimation  E(6-9)  , and  it  is  well  known  that 


•.  ! 
;■  I 
• r> 


0 = E(0|X^,  teS)  . 

/N 

In  general,  0 is  extremely  difficult  to  determine.  However,  if  X is  a SIP,  we 
have  a complete  solution.  In  formulating  the  main  result  we  will  use  the  notation 
of  Section  2,  and  identify  L2(X)  with  L2(A)  ® (pfQH*^(X))  by  Theorem  2.4.  We 
let  b2(X;S)  = L2(X^,  teS)  and  H(X;S)  = H(X^,  teS) . 


THEOREM  4.1  Let  \ be  a nondegenerate  SIP  (0,R; F)  with  F(0+)  = 0 and  let 


8eL^(X)  have  the  following  orthogonal  development 


e ^ I 


n>l  p.+ 

p>0  Yj< 


V Pr--Pk  ® ~ ®Pk 

••+P^=P  1"  k " 4 \ 
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Suppose  that  (X^,  teS)  is  nondegenerate.  Then 


9=1  I 


Pl---Pk 


n>l  Pj+...+p.=p  " ^r--\ 
P>0  Yj<...<y]J 


..Y  ® »•••»  ) 


where  for  ^cH(X), 


^ (=E(C|X^.  t.S))  . 

Proof.  Recall  the  definition  of  the  r.v.  A and  observe  that  it  is  independent 
of  the  choice  of  the  defining  sequence  {?.}.  It  then  follows  that  AfL,(X;S) 
and  every  peL2(X;S)  has  the  orthogonal  development 


P=  I I 


qiO  q +...+q  =q- 

m>l  (5, <...<6^ 

1 J 


« (n^  J)  . 


where  {p^,  ficA}  (A  linearly  ordered)  is  a CONS  in  H(X;S). 


We  have 


0 - E(6|X  , teS)  = Proj  0 

- y y Pi"'Pk„  . ~ *Pl 

P^o  P -p  n yy-yy  n^  S.  ’ 

n>o  y;<...<)|; 


Thus  to  show  the  theorem  it  suffices  to  show  that 


’’^°JL„(X;S)®n^^^  ® ®- • -®  C ) = e (A)  B (I  ? ^)  . 

1 'k  'k 

If  Cj , . . . ,C  eH(X)  and  n, , . . . ,n  €H(X;S)  then  it  follows  from  <£  ,n  > = 


i’  'j  H(X) 


<C,  ®...B  c . n S...S  n > 5 = <t  S...S  I q £...£  q > 

Ph'(X)  pi  P H®P(X) 

and  hence  for  each  peL.,(X;S)  we  have 
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~ »P,  ^ »Pi 


_ V ^1  ~ ^ ®Pfc  »q,  ^ ^ ®q. 

■ 6j...6^‘^%^A)  » (5^^  8. ..8  ).  e^(A)  8 (n^^l  ®...8 


(X) 


- fA)  8 it 


®Pi~ 


m <5, . . .6  . n' 
1 J 

<.»Pl 


~ ®Pk  ®^i  ~ ~ *^1; 

••®  ).  e (A)  8 (n^  8... 8 Hr  ^)> 


S.  ^'L^CX) 


<e„(A)  8 (F  B...B  F ").  p> 


.®P 


1 


'1 


L2(X)  • 


Since  e^(A)  8 * 8...e  F '')  c L2(X;S),  the  result  follows. 

'1  'k 


0 


A similar  result  can  be  obtained  when  F(0+)  > 0.  Of  particular  interest  is 
the  case  where  X is  in  fact  a Gaussian  process.  In  this  case  we  have  A = 1 = 
^2^^^=  p>o  Qp  = *^*^(X);  and  Theorem  4.1  yields  the  following. 


CpfK)LLAItY  4.2  Let  X be  a GP(O.R)  and  let  6 e L^CX)  have  the  following 
orthogonal  development 


e = I I 


p>0  p +.  . .-fp  =p  ^l---\  \ 


Then 


Pl...Pk  «p  8p^ 


6 = I I a‘  ■'r  ‘»...®C 

p>0  Pj  + . . .♦Pk^p  'k  '1 


where  for  F e M(X), 


" '’''°jH(X;S)^  (=E(C|X^,  teS))  , 


Consequently, 


rrojjj^  . 


1 


I . I 


I 1 


L 
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where  Qp(S)  denotes  the  p-th  homogeneous  chaos  of  (X^,  teS)  . 

COROLLARY  4.3  If  X is  a nondegenerate  SIP  (0,R;F)  and  if  Jc^^dF(a)  < «> 
for  allt  e P , then  for  all  ^eH(X), 


HCH 


|X^.  teS)  = H 


njrP2 


,(U  , 


E(exp{^  - ^ E5^}|X^,  teS)  = exp{C  - EC") 


In  particular,  if  X is  a GP(0,R)  and  5eH(X),  then 


■ "p,E5=<5>  • 


E(exp(5  - liE5^)|Xj,  teS)  .exp{?  - !jE5‘^) 


If  X is  a zero  mean  Gaussian  martingale  then 

"p,EX*'*t’- 


are  martingales. 


Proof:  Only  the  first  assertion  requires  proof  and  we  shall  prove  it  for 

the  case  F(0+)  = 0 only.  /e^“dF(a)  < ® , t e IR , implies  L,(A).  Thus 

E 


E 

u °^i  2 


p.-^E 


(E) 


1 


'■A'  A 5 

p.  A p 

1 

E 

, 2 Bp  ~ 

= (pn'^c—)  e C e L,fA)  B H*P(X)  c L2(X) 


(by  (2.4)) 


1 


and  by  Theorem  4 . 1 


E 

2 


01  = (P-')’^(/)'  B = H 


(O  . 


p.-^c 

* Oj 


Now  write 


t.  -1  <»  ■ *J 
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- 537  • L 

1 p>o  v>7r^V 


(O  , 


the  convergence  of  the  series  being  pointwise.  It  follows  from  the  expression 

of  9^  that  all  terms  in  the  above  sum  are  mutually  orthogonal  and  each  term 

has  square  norm  -2  • i ^ , where  a = Ja^dF(a).  Therefore  the  series  con- 
(jP  p-  p 

verges  also  in  L2(X)  and 


p>0  P-  p>0  P- 

1 1 


I A 

p^o 

1 


= -Pf^  - 2^  ES^}  . 


For  X a Wiener  process  it  is  well  known  that  (X^-t,  t>0)  and 
(exp{X^  - t>0)  are  martingales. 

If  X is  a zero  mean  Gaussian  process  and  T = (or  any  interval) 

then  by  Corollary  4.3  we  have  that  for  all  s<t  , 


S'"  „2<V  -2 

P.EX,  P.EKt.s 


where 


^,s  = • 

An  expression  for  X can  always  be  obtained  via  the  Cramer-Hida  representation 

L j S 


of  X: 


Then  we  have 


= I f^''\t.u)dZ^''^ 

n=l  -«> 


Xt  s = i / f^"^t.u)dzf’‘ 

n=l  ™ 


■ .'’V  V' 
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2 2 

The  case  with  p = 2,  i.e.  the  L2-functional  , is  considered  in  Hida  and 

Kallianpur  (1975)  for  a special  class  of  Gaussian  processes  X.  It  should  be 

/N 

clear  that  whenever  a simple  expression  is  available  for  X , then  a simple 

T , S 

expression  is  also  found  for  the  nonlinear  predictor  of  H ^(X.), 

P,EXJ 

We  close  this  section  with  a simple  example.  Let  (X^,  -<»<t<“)  be  a station- 

2 

ary  Gaussian  process  with  EX^  = 0,  EX^  = 1,  and  continuous  covariance  R(t),  which 
is  reciprocal  or  quasi-Markov  on  [0,T] . Then  it  is  known  [ 13,  5] 
that  R(t),  OstsT,  has  one  of  the  following  three  forms;  Ae  + (l-A)e^^  where 
a > 0,  A > *2,  T < (2a)  ^log(A/ |A-1  | ) ; cos  at  where  a > 0,  TSir/a;  1-at  where 
0 < a < 2/T.  We  want  to  estimate  0,  an  L2-functional  of  X^,  0<u  <t  <v<T, 
based  on  observations  X^,  seS  = [0,u]u[v,T].  Since  X is  reciprocal  or  quasi- 
Markov  on  [0,T]  we  have 

X.  = E(XJX  , scS)  = ctX,  + ex^ 

V.  u s U V 

where 


R(u-t)-R(v-t)R(u-v) 

l-R^(u-v) 


g ^ R(v-t)-R(u-t)R(u-v) 
l-R^(u-v) 


Since  0 is  an  L2“functional  of  X^,  it  has  the  orthogonal  development 
0 = ^p^p^^t^’  thus  the  best  estimate  0 of  0 is  given  by 


0 = I 3pH  .2^\^  = ^ 2 2 ^”^u  ^ ■ 

p^O  ^ P.EX^  p>0  ^ p,a  +6'^+2agR(u-v) 


4 . 2 Nonlinear  Prediction 

Consider  the  following  prediction  problem:  Let  X = (X^,  teT) , T an  interval, 
be  a second  order  process  and  let  Y = (Y^  = 0^(X^),  teT)  with  0^  a real  function 
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2 

such  that  EY^  = 0 and  EV^  < <»  for  all  tcT.  Suppose  that  on  the  basis  of  the 
past  values  of  Y up  to  time  t we  want  to  find  the  best  prediction  of  the  future 
value  T > 0.  Two  predictors  are  of  special  interest:  the  optimal  linear 

predictor  Yj^(t,x)  and  the  nonlinear  predictor  Y^^(t,x).  The  optimality  is 
in  the  sense  of  minimizing  the  mean  square  error  within  the  class  of  all  linear 
and  nonlinear  predictors  respectively.  It  is  well  known  that 


Y „(t,T)  = E{Y  Y , s<t)  , Y„(t,T)  = Proj„,„  ,Y^  ^ 

nil  t+x'  s’  ^ ' H(Yg,s<t)  t+x 


The  corresponding  mean  square  predictor  errors  are  denoted  by 


, o2(t,T)  . , 

Now  introduce  a "super  predictor"  Y^(t,x)  as  the  nonlinear  predictor  of 
based  on  (X^,  s<t) , i.e. 


. £(Vj.,|X3,  ssti  , 


and  denote  its  mean  square  error  by  a^(t,x). 


It  is  clear  that 


(4.1) 

2 

and  thus  provides  a lower  bound  for  the  mean  square  errors  of  linear  and 

nonlinear  prediction.  If  X is  a SIP  then  can  be  obtained  as  in  Section 

4.1  by  solving  an  estimation  problem.  If,  in  addition,  0^  is  one-to-one 

for  each  t then  the  a-fields  generated  by  and  Y^  coincide.  In  this  case 

Ynil(t'X)  = Y^(t,x)  = ^(^^.+^1^5*  nonlinear  predictor  can  be 

again  obtained  by  solving  an  estimation  problem. 

In  the  important  case  where  X = (X^,  t e IR  ) is  a zero  mean  stationary 

Gaussian  process  with  covariance  function  R(t,s)  = R(t-s)  and  9^  = 0 we  can 

2 2 

calculate  the  lower  bound  a (t,x)  = o (x)  as  follows.  Write 

s s 


1 


I 

! 

1 

I 

■ I 
1 
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00 

(4.2)  Y,  . e(X^)  = I 4 H (X^, 

P=1  ^ 

where  = EX^.  Note  that  for  all  C,neH(X), 

EHp^^^2(C)Hp^g^2(n)  = p!  < ?®P.n^>  = p!{E(Cn)}P 

° ifpM. 

Thus  we  have  • | 

(4.3)  EY^Yg  = I p!aV(t-s)  . .: 

p^l 

Let  X(t,T)  = optimal  nonlinear  predictor  of 

2 

(which  is  also  the  optimal  linear  predictor  since  X is  Gaussian) .and  Oq(t)  i 

be  the  mean  square  error.  Then  by  Corollary  4.3, 

(4.4)  Y (t,T)  = I a H ^2  (X(t,T)) 

p>l  P p.EX^(t.T) 

and  hence 

(4.5)  • I plaVP  - I p!a^|a^-o^(t))>’ 

P>1  P P>1  P ^ 

= I pla^^o^P  - [o^-Oq(t)]P}  . 

p>l  P 

It  is  well  known  from  the  general  theory  of  stationary  processes  (e.g. 

2 

Doob  (1953),  Rozanov  (1967)]  that  cJq(t)  can  be  obtained  analytically  (if  not 

explicitly)  through  the  Wiener-Paley  factorization  theorem  if  X is  linearly 

regular,  i.e.  n^H(X^,  sst)  = {O}.  When  X is  mean  square  continuous  and  linearly 

2 

regular  we  now  show  that  so  is  Y,  and  hence  c^([^(t)  can  be  obtained  analytically. 

Y is  clearly  stationary  and  its  mean  square  continuity  follows  from  the  con- 
tinuity of  R and  (4.3).  The  linear  regularity  of  Y follows  from  the  fact  that 


I*' 


V 
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a Gaussian  process  is  linearly  regular  if  and  only  if  its  remote  past  is 

trivial.  Here  we  give  a purely  geometric  proof  of  this  property;  for  a proof 

using  Kolmogorov's  zero-one  law  see  Rosanov  (1967)  and  Ibragimov  and  Linnik 
(1971) . 

THEOREM  4.4  Let  X = (X^ , teIR)  be  a zero  mean  Gaussian  process  . 

®t  " and  = n^B^.  Then  X is  linearly  regular  if  and  only 

if  is  trivial. 

Proof:  The  "if"  part  is  clear.  For  the  "only  if"  part,  note  first  that  the 

triviality  of  B_^  is  equivalent  to  the  condition 

For  simplicity  we  write  = L2(X^.  s<t)  and  = H(X^,  s<t) . Thus  we  need 
to  show  that  = {0}  implies  L_^  = = {O}.  So  assume  that  = {0} 

and  let  We  will  show  that  6=0. 

Fix  a sequence  {tj^.  nsl.2,...)  decreasing  to  The  family  of  subspaces 

H has  the  property  that  c H for  m < n and  n H . {o).  Thus  it  follo.s 

m n •'n 

readily  (see  e.g.  Rozanov  (1967),  pp.  53.  56)  that  lim  Proj^  C = 0 for  all 

n 

CeH(X) , and  hence  ^ 

00 

H.  = ® D 
h n=l  " 

where  is  the  orthogonal  complement  of  H in  H.  , i.e.,  H = H ® D . 

n+1  n ^n+1  " 

In  each  pick  a CONS;  then  pool  all  these  CONS’s  together  to  get  a CONS 

in  H . Observe  that 
1 

(4.6)  ° • 

"n 

Since  0€L__^  orthogonal  development 


e=  I I 


Pi---Pk  »Pi 

j c 


p>0  p +. . .+p.  =p  ’1' ■ ■ 'k  '1 


and  by  Corollary  4.2, 


0 = Proj,  % = II  E ) 


Yi-.-Yk' 


n 


..®(Proj  E) 

\ \ 

n 


which  together  with  (4.6)  yields 

- p. . . .p  ®p  « 

(4.7)  E0^  = n Y)ll(Proj  £ ) ^ S. . .S(Proj  £ ) 

’l"'\  "t  4 t ^k 


»t  ^1^ 
n 


Now  let  n ->•  “ in  (4.7).  The  limit  can  be  taken  inside  the  summation  since 

each  summand  is  bounded  by  jj£  ®...»  £ ^| | ; and  the  limit  of  each 

’1  ^k 

2 

summand  is  zero  because  of  (4.6).  Consequently  E0  = 0,  i.e.  0=0,  and  the 
theorem  is  proved.  □ 


Jaglom  (1970)  has  considered  the  problem  of  comparing  the  performance  of 

optimal  linear  and  nonlinear  predictors  for  polynomial  functions  of  certain 

stationary  Markov  processes.  Donelson  and  Maltz  (1972)  studied  this  problem 

in  detail  for  polynomial  functions  of  the  Ornstein-Uhlenbeck  process.  The 

inequality  (4.1)  plays  a central  role  in  such  studies.  As  an  example,  let  X 

be  the  Ornstein-Uhlenbeck  process,  i.e.  a zero  mean  Gaussian  process  with 

- 1 t-s I 

covariance  function  R(t,s)  = e ' and  let  Y be  given  by  (4.2).  By  the 

Markov  property  of  X we  have 

X(t,T)  = E(X^^^(X^,  s<t)  = e"'^X^  . 

Thus  it  follows  from  (4.4)  and  (4.5)  that 


Y3(t,T)  = lali  .2t  ’ 

p>l  ^ p,e  p>l  P P 


•’W  * •• 
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